Matching-adjusted indirect comparison of phase 3 clinical trial outcomes of OC-01 (varenicline solution) nasal spray and lifitegrast 5% ophthalmic solution for the treatment of dry eye disease

BACKGROUND: Matching-adjusted indirect comparison (MAIC) is a methodology for cross-study comparisons after adjusting for baseline characteristic imbalances. It is a comparative analytical approach used across therapeutic areas absent head-to-head trial outcomes. OBJECTIVES: To compare the efficacy of OC-01 (varenicline solution) 0.03 mg nasal spray (OC-01 VNS) to lifitegrast 5% ophthalmic solution on tear production and patient-reported eye dryness in patients with dry eye disease (DED) using data from phase 3 clinical trials via MAIC analysis. METHODS: Individual patient data (IPD) from the phase 3 registrational trial of OC-01 VNS and aggregate data from 2 phase 3 trials of lifitegrast in the publicly available XIIDRA New Drug Application were used. Using unanchored MAIC methods, IPD were weighted on clinically relevant baseline variables (age, race, sex, baseline Schirmer’s test score [STS], and Eye Dryness Score [EDS]) to produce weighted OC-01 VNS datasets matched to the same lifitegrast datasets’ variables. Least-squares (LS) mean change from baseline (CFB) in STS for OC-01 VNS was calculated using the identical analysis of covariance model and covariates used to calculate the same values for lifitegrast in the XIIDRA New Drug Application and was then compared. LS mean EDS (based on a 100- point Visual Analogue Scale) was compared via analysis of covariance in the weighted OC-01 VNS and lifitegrast datasets. OC-01 VNS at 2 and 4 weeks compared to lifitegrast data at 2 and 6 weeks. RESULTS: Data from 511 subjects (n = 260 treated; 251 vehicle control [VC]) in the OC-01 VNS phase 3 trial, 588 (n = 293 treated, 295 VC) in the lifitegrast phase 3 OPUS-1 trial, and 718 (n = 358 treated, 360 VC) in the lifitegrast phase 3 OPUS-2 trial were analyzed. The LS mean STS CFB for OC-01 VNS at 2 and 4 weeks was significantly greater than that for lifitegrast at 2 and 6 weeks in OPUS-1 and OPUS-2 (P < 0.0001 for all comparisons). The LS mean EDS CFB for OC-01 VNS at 2 and 4 weeks was significantly greater than that for lifitegrast at 2 and 6 weeks in OPUS-1 (P < 0.0001 for both comparisons) and at 4 weeks vs lifitegrast at 6 weeks in OPUS-2 (P < 0.0001). CONCLUSIONS: This MAIC analysis demonstrates OC-01 VNS produced significantly greater improvement in mean STS and comparable or greater improvement in EDS compared with lifitegrast in phase 3 trials. These findings suggest a potentially greater magnitude of improvement achieved with OC-01 VNS compared with lifitegrast for the treatment of DED within the conditions of the analysis methodology.


METHODS:
Individual patient data (IPD) from the phase 3 registrational trial of OC-01 VNS and aggregate data from 2 phase 3 trials of lifitegrast in the publicly available XIIDRA New Drug Application were used. Using unanchored MAIC methods, IPD were weighted on clinically relevant baseline variables (age, race, sex, baseline Schirmer's test score [STS], and Eye Dryness Score [EDS]) to produce weighted OC-01 VNS datasets matched to the same lifitegrast datasets' variables. Least-squares (LS) mean change from baseline (CFB) in STS for OC-01 VNS was calculated using the identical analysis of covariance model and covariates used to calculate the same values for lifitegrast in the XIIDRA New Drug Application and was then compared. LS mean EDS (based on a 100point Visual Analogue Scale) was compared via analysis of covariance in the weighted OC-01 VNS and lifitegrast datasets. OC-01 VNS at 2 and 4 weeks compared to lifitegrast data at 2 and 6 weeks.

Plain language summary
Dry eye disease is often caused by reduced tear production. We compared data from previous studies of OC-01 (varenicline solution) 0.03 mg nasal spray (OC-01 VNS) and lifitegrast 5% ophthalmic solution for dry eye treatment. Within the conditions of the analysis, OC-01 VNS increased tear production superiorly compared with lifitegrast in both trials and improved patient eye comfort (symptom) outcomes comparably or more than lifitegrast. For patients with dry eyes, OC-01 VNS may produce more natural tears than lifitegrast.

Implications for managed care pharmacy
This matching-adjusted indirect comparison of data from phase 3 trials of OC-01 VNS and lifitegrast ophthalmic solution for dry eye demonstrated significantly greater improvement in tear production with OC-01 VNS, as compared with lifitegrast. Absent head-to-head data, this analysis suggests that OC-01 VNS provides greater improvement than lifitegrast for dry eye disease.
Dry eye disease (DED) is a common ocular condition frequently associated with significant adverse effects on visual function. It is multifactorial and characterized by a persistently unstable or deficient tear film. 1 The longitudinal Beaver Dam Eye Study reported the prevalence of DED (defined by self-reported symptoms within prior ≥3 months) to be 17.2% in men and 25.0% in women. 2 DED adversely effects quality of life. [3][4][5][6] Patients with DED report poorer self-perceived health status and more psychological stress, 7 activity limitations, 4 worse mental health, role difficulties, and driving issues 8 than those without DED. Increased DED severity worsens quality of life. 8 DED imposes a significant economic burden. In Canada, the total cost of DED (including both direct and indirect costs) is estimated at more than $24,000 CAD annually per patient, most of which (79%) is attributable to reduced productivity at work rather than absenteeism from work. 8 In the United States, the estimated direct costs of DED to the health care system are nearly $4 billion USD annually and costs increase with disease severity. 9 The complex and multifactorial nature of DED precludes consensus on a treatment cascade: therapy is individualized based on contributory factors in individual patients. 10 When conservative measures are exhausted, pharmacological therapy is warranted. Topical corticosteroids can be useful for short-term use during acute exacerbations although long-term use is not indicated because of the risk of complications such as elevated intraocular pressure and cataract formation. 10,11 Pharmacologic options include cyclosporine A 0.05% (RESTASIS, Allergan an AbbVie company) 12 and 0.09% (CEQUA, Sun Pharma), 13 lifitegrast (XIIDRA, Novartis), 14 and most recently in October 2021, OC-01 (varenicline solution) 0.03 mg nasal spray (OC-01 VNS) (Tyrvaya, Oyster Point Pharma, Inc.). The former 2 drugs act through inhibition of T lymphocyte-mediated inflammation (cyclosporine is a calcineurin inhibitor, 15 whereas lifitegrast is a lymphocyte function-associated antigen 1 antagonist 16 ). OC-01 VNS is a highly selective nicotinic acetylcholine receptor agonist that promotes tear production through activation of the trigeminal parasympathetic pathway when administered as an intranasal spray. 17 Varenicline has been approved in the United States for smoking cessation via oral tablet administration since 2006 and it safety profile is well established. 18 OC-01 VNS 0.06 mg (twice the approved OC-01 VNS dose) showed significantly less systemic absorption than oral dosing, with no new tolerability signals emerging. 17 In phase 2b and 3 studies, a 10 mm or more increase in tear production measured by Schirmer's test score was achieved by 47-52% of patients treated with either OC-01 VNS 0.03 mg or OC-01 VNS 0.06 mg for 28 days compared with only 14-28% of patients receiving vehicle control. 19 No randomized head-to-head comparative trials of DED drugs are available to inform managed care entities and clinicians. We performed a matching-adjusted indirect comparison (MAIC) of OC-01 VNS and lifitegrast. The validated MAIC methodology facilitates cross-study comparisons after adjusting for key baseline characteristic differences and are used by global health agencies (eg, England's National Institute for Health and Care Excellence) as part of new drug submissions. 20,21 MAIC involves comparing individual patient data (IPD) from a clinical trial to a publicly available aggregate clinical trial dataset. Patients in the IPD dataset are individually weighted such that the resulting IPD adjusted dataset matches the aggregate data (AD) dataset on key demographic and baseline factors to minimize bias. An anchored MAIC requires evidence connected by a common comparator, whereas unanchored MAIC does not. 22 MAICs have been used across therapeutic areas, including cancer, 23 cardiovascular disease, 24 vaccines, 25 and a recent comparison of therapies for age-related macular degeneration. 26 We report an unanchored MAIC of OC-01 VNS 0.03 mg and lifitegrast in patients with DED.

Methods
We conducted an unanchored post hoc MAIC of deidentified IPD derived from a vehicle-controlled phase 3 clinical trial of OC-01 VNS reported in part by Wirta 27 and deidentified AD from 2 vehicle-controlled phase 3 trials of lifitegrast 5% (OPUS-1 and OPUS-2) obtained from the publicly available XIIDRA New Drug Application (NDA) Statistical Review 14 and reported in part by Sheppard et al (OPUS-1) 28 and Tauber et al (OPUS-2). 29 The MAIC was unanchored because of vehicle control route of administration differences: (nasal spray vs eye drop). Comparison with OPUS-3 study was not possible because Schirmer's test score (STS) was not an outcome in OPUS-3. 14,30 Key design elements are presented comparisons). The LS mean EDS CFB for OC-01 VNS at 2 and 4 weeks was significantly greater than that for lifitegrast at 2 and 6 weeks in OPUS-1 (P < 0.0001 for both comparisons) and at 4 weeks vs lifitegrast at 6 weeks in OPUS-2 (P < 0.0001).

CONCLUSIONS:
This MAIC analysis demonstrates OC-01 VNS produced significantly greater improvement in mean STS and comparable or greater improvement in EDS compared with lifitegrast in phase 3 trials. These findings suggest a potentially greater magnitude of improvement achieved with OC-01 VNS compared with lifitegrast for the treatment of DED within the conditions of the analysis methodology. evaluated in ONSET-2 and the OPUS-1 and OPUS-2 studies; however, differing corneal region assessment with differing scoring systems were used (ONSET-2 used the NEI/ Industry Workshop 0-3 Scale; OPUS-1 and OPUS-2 used the ORA 0-4 Scale 28 ), precluding comparisons between studies.
Mean change from baseline in STS and EDS were analyzed using analysis of covariance (ANCOVA) models to estimate treatment effects for each treatment vs its vehicle control. The ONSET-2 dataset was weighted separately to the OPUS-1 and OPUS-2 datasets. Given differences in time points between studies, outcomes at 2 weeks were compared between studies and outcomes at 4 weeks in the ONSET-2 dataset were compared with outcomes at 6 weeks in the OPUS-1 and OPUS-2 datasets. Least-squares mean treatment differences (active treatment minus vehicle control) with 95% CIs in the OPUS-1 and OPUS-2 datasets were extracted from the XIIDRA NDA. 14,35 These were derived from ANCOVA models with treatment, artificial tear use within 30 days of screening, baseline inferior corneal fluorescein staining, and baseline EDS or STS, as reported in the XIIDRA NDA. 14,35 To ensure comparability of statistical methods, least-squares mean treatment differences (with 95% CIs) for ONSET-2 were derived from a similar weighted ANCOVA model using the same covariates. These leastsquares means for ONSET-2 and OPUS-1 and OPUS-2 were then compared using t-tests. For all analyses, missing data for both studies were imputed using the last observation carried forward method, and all statistical comparisons were 2-sided with α = 0.05.

Results
The ONSET-2 dataset used for this analysis included all subjects enrolled in the OC-01 VNS 0.03 mg group (n = 260) and in the vehicle control group (n = 251). The OPUS-1 datasets included all subjects enrolled in the lifitegrast 5% group (n = 293) and in the vehicle control group (n = 295), and the OPUS-2 dataset included all subjects enrolled in the lifitegrast 5% group (n = 358) and in the vehicle control group (n = 360). Demographic and baseline data from the unweighted and weighted ONSET-2 and OPUS-1 and 2 datasets are given in Table 1.

STS MEAN CHANGE FROM BASELINE
ONSET-2 Weighted for OPUS-1. In the ONSET-2 dataset weighted for OPUS-1 at 2 weeks, ANCOVA least-squares mean improvement in STS was 11.24 mm (95% CI = 8.62 to 13.85) in the OC-01 VNS group vs 5.07 mm (95% CI = 2.64 to 7.50) in the vehicle control group, a treatment effect of 6.17 mm (95% CI = 4.58 to 7.75; P < 0.0001), favoring OC-01 VNS in Supplementary Table 1 (available in online article). The OC-01 VNS phase 3 study and the OPUS-1 and OPUS-2 phase 3 studies included subjects with baseline STS 10 mm or more, but OPUS-2 restricted entry to subjects with more severe DED (Eye Dryness Score [EDS] >40). 14 The methodology of MAIC analysis has been previously reported. 21,31 The IPD from the OC-01 VNS dataset were matched to the AD of the lifitegrast OPUS-1 dataset on 4 demographic or baseline characteristics common to both datasets: age, sex, race, and baseline STS. EDS was considered a putative matching variable for comparison with the OPUS-1 dataset; however, inclusion of EDS in the MAIC analysis greatly reduced the effective sample size of the weighted OC-01 VNS data by nearly 50% (Supplementary  Table 2), a well-described limitation of MAIC analysis. [20][21][22]32 Subjects in the OC-01 VNS dataset were weighted by their predictive probability of inclusion in the OPUS-1 dataset, which produced an adjusted OC-01 VNS dataset matched to the OPUS-1 dataset on the 4 prespecified baseline variables. MAIC uses regression modeling to estimate propensity scores via generalized method of moments. 33 Subjects in the treatment and vehicle control groups were weighted separately at the 2-and 4-week time points so that only subjects with complete data at each time point were included. The analysis used R software package (version 4.0.3), with the "maic" package used to calculate IPD weights. 34 IPD from the OC-01 VNS dataset were matched to the AD of the lifitegrast OPUS-2 dataset on 5 demographic or baseline characteristics common to both datasets: age, sex, race, baseline STS, and EDS. Matching was undertaken with the OC-01 VNS data from the ONSET-2 dataset to include only subjects with baseline EDS of 40 or more to better match the OPUS-2 dataset. This significantly reduced sample size loss when including EDS as a weighting factor in the MAIC ( Table 3).
The primary outcomes evaluated in this MAIC were mean change from baseline in STS and the mean change from baseline in EDS, as these were common between the OC-01 VNS and lifitegrast datasets. Of note, STS was performed without anesthesia in OPUS-1 and OPUS-2 and with anesthesia in ONSET-2; although the effect of reflex tearing might result in greater unanesthetized vs anesthetized STS, changes from baseline should not be significantly affected by this difference in methodology, as the consistent measurement methodology likely cancels out in the change-from-baseline calculation, although there is no formal evidence on this point. EDS was assessed using a 100-point Visual Analogue Scale in which subjects drew a line along the spectrum from 0 (no dryness) to 100 (extreme dryness) to indicate their instantaneous perception of their eye dryness. Ocular surface staining was mean STS improved by 1.75 mm (95% CI = 1.16 to 2.34) in the lifitegrast group and by 1.47 mm (95% CI = 0.88 to 2.06) in the vehicle control group, a treatment effect of 0.29 mm (95% CI = −0.52 to 1.10; P = 0.4882), indicating no significant difference between lifitegrast and vehicle in OPUS-1.

FIGURE 2
Mean Change From Baseline Schirmer's Test Score for OC-01 VNS ONSET-2 and Lifitegrast OPUS-2 and at 2, 4, and 6 Weeks and significant at both time points in the OPUS-2 lifitegrast dataset. The Statistical Review of the XIIDRA NDA identifies inconsistent effects on EDS across studies, with EDS improvements demonstrated at all time points in OPUS-2 and -3 in eyes with more severe disease (EDS > 40 at baseline), 14 as expected given the greater therapeutic window when starting with a higher baseline value. A recent post hoc pooled analysis of OPUS-2 and -3 demonstrated an insignificant likelihood of experiencing an improvement of 30% or more in EDS if baseline EDS was 60 or more compared with a significant likelihood of improvement if baseline EDS was more than 60, 37 suggesting lifitegrast's efficacy is greater in eyes with moderate to severe DED. Given the disease severity and response to therapy relationship, EDS was a weighting factor in both MAIC comparisons, and EDS outcomes showed significance with OC-01 VNS compared with lifitegrast at both time points in OPUS-1. When data were subset to include only patients with an EDS of 40 more to match the OPUS-2 study, significance was demonstrated at the later time point. The vehicle controls in these 2 studies differed significantly. The OPUS-1 and -2 vehicle was a topical ophthalmic solution that may potentially provide some (non-placebo) effect per application of liquid to the ocular surface, although this manifested only for EDS in the OPUS-2 dataset. The control in the ONSET-2 trial was a nasal spray consisting of the OC-01 VNS vehicle. Mechanical stimulation of the anterior ethmoid in the nasal mucosa may momentarily, on contact, stimulate the trigeminal parasympathetic pathway. [38][39][40][41] The significantly greater benefit of pharmacologic neuroactivation with OC-01 VNS over its vehicle, using the same optimized nasal route of administration, demonstrates the additional DED and may not directly affect tear production (although STS was a prespecified study outcome in a phase 2 and in 2 phase 3 lifitegrast trials). The Statistical Review section of the XIIDRA NDA showed no statistically significant effect of lifitegrast on STS at any time point in a phase 2 study or in the phase 3 OPUS-1 and OPUS-2 studies (STS was not a measured outcome in the phase 3 OPUS-3 study). 14 EDS is a patient-reported outcome that assesses symptomatology and more directly characterizes patients' perception of treatment efficacy. Compared with respective vehicles, EDS improved significantly with OC-01 VNS at both time points in the OPUS-1 and OPUS-2 weighted ONSET-2 datasets, and lifitegrast was unchanged compared with vehicle at both time points in the OPUS-1 dataset improves signs and symptoms of DED, as compared with lifitegrast.
Outcomes assessed in this MAIC were restricted to those common to both data sets. DED is characterized by insufficient or unstable tear production. STS measures tear flow over 5 minutes. When performed with anesthesia, it isolates basal tear secretion by minimizing reflex tearing. US Food and Drug Administration guidance on DED trial design allows for anesthetized or unanesthetized STS as a study endpoint and proposes an improvement of 10 mm or more as a clinically meaningful outcome to support the indication for the treatment of the signs and symptoms of DED. 36 OC-01 VNS activates natural tear production, as demonstrated by anesthetized STS improvement, whereas lifitegrast targets the inflammatory aspect of

FIGURE 3
Mean Change From Baseline Eye Dryness Score for OC-01 VNS ONSET-2 and Lifitegrast OPUS-1 and at 2, 4, and 6 Weeks

LIMITATIONS
The quality of source studies determines the validity of MAIC analyses. We included the most robust data available-from phase 3 randomized trials, including all participants in the treatment and control groups. MAICs cannot control for differences in unobserved variables across studies and only evaluate outcomes quantified similarly between studies. MAICs cannot adjust for differences in time points assessed across studies. We best matched disparate time points by comparing the common 2-week time points and the 4-week OC-01 VNS time point to the 6-week lifitegrast time point to minimize bias, as DED is chronic but generally stable. 44 Although MAIC can minimize key differences in baseline variables between datasets, attempts to minimize very large differences can produce weighted datasets with small effective sample sizes (resulting from low-or zero-weighting of noncomparable subjects) and preclude meaningful comparisons. This occurred when attempting to weight the ONSET-2 dataset using EDS in OPUS-1 (Table 2) but not OPUS-2 ( Table 3). The reduction in effective sample size with the OPUS-1 analysis occurred because of different mean baseline EDS scores between the datasets, and there were no restrictions on baseline EDS values in either study's dataset eligibility criteria, precluding subsetting the ONSET-2 data as done in the OPUS-2 analysis. Differences in mean baseline EDS between ONSET-2 and OPUS-1 datasets could not be mitigated using statistical means. Additionally, comparison with OPUS-3 study was not possible because STS was not an outcome of OPUS-3.

Conclusions
Comparative efficacy is ideally evaluated in head-to-head randomized been applied to ophthalmology in a comparison of aflibercept and ranibizumab treat-and-extend strategies for neovascular age-related macular degeneration across 6 randomized trials. 26

STRENGTHS
The strength of the MAIC approach is the ability to conduct cross-study comparisons by minimizing differences in baseline study sample characteristics. This provides an estimate of relative efficacy for drugs absent head-tohead trials and can inform decisions by physicians, patients, and entities that determine coverage/formulary inclusion. MAIC analyses are inexpensive and use existing data, precluding the need for cooperation among competing organizations.
magnitude of natural tear production effects of active cholinergic agonist receptor activity, resulting in significantly improved tear production and symptomatic outcomes compared with vehicle. MAIC as a statistical tool for comparative evaluations is a relatively new methodology first described in 2010 42 and rapidly adopted: of a total of 170 results of a PubMed search ("matching-adjusted indirect comparison"), more than 50 were published in 2021 alone. The value of MAIC is recognized by the pharmaceutical industry (as many PubMed-listed studies are industry-sponsored), as well as global health agencies such as the England's National Institute for Health and Care Excellence 20,31 and similarly in France, Germany, and Canada. 43

FIGURE 4
Mean Change From Baseline Eye Dryness Score for OC-01 VNS ONSET-2 and Lifitegrast OPUS-2 and at 2, 4, and 6 Weeks trials, but in their absence, physicians, patients, payers, and formulary regulators rely on available evidence in selecting therapy and making policy decisions. This MAIC represents the most robust comparison to date regarding the relative efficacy of OC-01 VNS and lifitegrast for the treatment of DED. We ensured complete transparency regarding the selection, nature, and sources of data included and replicated the analytic techniques, modeling strategies, and covariates used in the XIIDRA NDA to provide the most accurate phase 3 vs phase 3 comparison of available relevant data. The efficacy analysis presented herein demonstrates OC-01 VNS in the ONSET-2 trial produces significantly greater improvement in natural tear production (as measured by STS), as compared with lifitegrast in the OPUS-1 and OPUS-2 studies and significantly greater symptomatic improvement in mean EDS compared with lifitegrast in OPUS-1 and at the later time point in OPUS-2. Findings suggest patients who suffer from DED may experience a potentially greater magnitude of early improvement in tear production and symptoms when treated with OC-01 VNS compared with lifitegrast, as shown within the conditions of the MAIC analysis methodology.